Markov chains and decision processes for congestion avoidance and power control. (Chaînes de Markov et processus de décision markoviens pour le contrôle de congestion et de puissance)

This thesis is based on some applications of Markov chains and decision processes for performance analysis ofcongestion avoidance algorithms and power control. First, we study the behaviour of the window size of an MIMDcongestion control algorithm when subject to different loss processes. We show that the logarithm of the embeddedwindow size follows an additive recursive equation, which can be modelled as a Markov chain. Through this model,we obtain the distribution of the window size and its moments when losses are independent of the window size. Wethen propose an approximation for the loss process in which each packet is lost with a constant probability, andshow that the throughput is inversely proportional to the packet loss probability. Next, we study a continuous timemodel for the window size of a general increase and instantaneous decrease congestion control algorithm. We showthat the window size is a Markov process for which we give the Kolmogorov equation for the stationary distributionfunction. By applying a transformation, we also show that the window size is equivalent to the workload in a queuewith state-dependent arrival and service rates. Through the Kolmogorov equation and the analogy with a queueingsystem we provide conditions under which two congestion control algorithms have related window size behaviour.For example, it is shown that the behaviour of the window size of the MIMD algorithm with a linear loss is similarto that of the window size of the AIMD algorithm with a constant loss intensity. Next, we model the instantaneousratio of the sending rates of two competing MIMD sessions as a Markov chain. Through this model we study thefairness properties of the MIMD algorithm when subject to different loss processes. For heterogeneous sources, weshow that the fairness index can be improved by introducing rate-dependent losses at an intensity which is greaterthan a certain threshold intensity. We also study the bandwidth sharing between AIMD and MIMD sources whenthe losses are synchronous. The results indicate that the AIMD sources obtain a rate that is independent of thelink capacity whereas the MIMD sources utilise the rest of the capacity. Next, we model the instantaneous andthe average queue sizes of a RED enabled queue as a non-homogeneous Quasi-Birth-Death process. In the limitwhen the averaging parameter goes to zero, we use the singular perturbation technique to find the joint distributionof the average and the instantaneous queue sizes. In this limiting regime, the expected average queue size can beapproximated by the solution to a fixed point equation. A problem related to energy-delay tradeoff in a wirelessdevice is studied next. The device has a finite energy battery using which it wants to transmit data. The batterycan recover some charge when left idle. In each slot, the device has to decide whether to transmit data or to leavethe battery idle in order to increase the battery lifetime. We formulate this problem as a Markov decision processand provide conditions under which the optimal policy is of threshold type. Finally, through simulations we studythe effect of two threshold based channel switching policies in UMTS on the expected delay performance of TCPsessions.